Vaccine safety monitoring in the age of the COVID-19 pandemic

Vaccine safety has long been a major topic on this blog for a number of reasons, not the least of which is to counter the longstanding efforts of the antivaccine movement to cast doubt on vaccine safety. Many health problems have been falsely linked vaccines such as MMR (and, previously, the thimerosal preservative that used to be in several childhood vaccines) and autism, or demonstrably incorrect claims that vaccines predispose babies to sudden infant death syndrome, or autoimmune disease, or infertility and premature ovarian failure in females, or obesity, and a whole host of other conditions, diseases, and problems up to and including cancer and death. It doesn’t matter that vaccines are effective and among the safest of medical interventions. (They have to be, given that they are administered to mass populations of children and adults without disease in order to prevent disease.) Indeed, the benefits of some vaccines go beyond even the diseases that they prevent, diseases that are more dangerous than most people appreciate, such as in the case of the measles vaccine. Most people are unaware of the multiple layers of systems monitoring vaccine safety and the National Vaccine Program Office (NVPO), the closing or “reorganization” of which has led me to great concern for the monitoring of COVID-19 vaccine safety.

Unfortunately, the success of the antivaccine movement in spreading fear, uncertainty, and doubt about vaccines on social media before social media companies started to crack down (and even after) has led to enough vaccine hesitancy that in a number of areas vaccine uptake has fallen to levels that endanger community immunity (formerly—and, sadly, still sometimes—referred to as herd immunity). Remember last year, when measles outbreaks due to decreasing MMR vaccination rates in various parts of the countries were a major concern, a problem that first burst into the US public consciousness after the Disneyland measles outbreak nearly six years ago? Soon after the pandemic hit, antivaxxers allied themselves with COVID-19 deniers, antimask and anti-lockdown protesters, and all manner of cranks and conspiracy theorists who have flourished during the pandemic and, for example, blamed the flu vaccine for increased susceptibility to COVID-19. This alliance is tight, up to and including the utterly bonkers conspiracy theory QAnon, which believes that President Donald Trump is battling a “deep state” group of Satanic pedophiles who harvest the blood of children for adrenochrome. (Don’t ask.) So it should come as no surprise that, as early as May (and earlier, ancient history in the timeline of the pandemic), antivaxxers started a preemptive disinformation war to cast doubt on any COVID-19 vaccines that will emerge from an unprecedented research effort. The Trump administration didn’t help, either. By naming its COVID-19 vaccine program “Operation Warp Speed,” it not only stoked doubt about whether any new vaccine would be adequately tested for safety, it made even staunch pro-vaccine advocates like me uncomfortable, as the Trump administration politicized the FDA and CDC to the point where I asked if we could trust them any more and, as Jann Bellamy wrote last week, the Government Accountability Office is investigating.

So I was not pleased when I read what happened in terms of vaccine safety monitoring last year.

Trump gutted the National Vaccine Program Office

It was with dismay that I read a story in The New York Times on Friday by Carl Zimmer reporting that last year the Trump Administration shut down a key office responsible for coordinating government programs monitoring vaccine safety and folded it into an office focused on infectious diseases. Noting that the first coronavirus vaccines will likely be approved next year, if not even earlier, and rolled out to millions of people in the US, Zimmer observes:

As the first coronavirus vaccines arrive in the coming year, government researchers will face a monumental challenge: monitoring the health of hundreds of millions of Americans to ensure the vaccines don’t cause harm.

Purely by chance, thousands of vaccinated people will have heart attacks, strokes and other illnesses shortly after the injections. Sorting out whether the vaccines had anything to do with their ailments will be a thorny problem, requiring a vast, coordinated effort by state and federal agencies, hospitals, drug makers and insurers to discern patterns in a flood of data. Findings will need to be clearly communicated to a distrustful public swamped with disinformation.

This has, of course, always been the most difficult issue when it comes to monitoring vaccine safety. People are pattern-seeking animals. We’re extremely good at noticing patterns and creating meaning or inferring causation from those patterns, even if the meanings or causations inferred are completely false. One commonly used example to illustrate this point is that if you ate some wild berries and then sickened, it would likely be beneficial to see the link between eating berries and illness, and then to avoid those berries in the future, even if the berries were not, in fact, the cause of the illness. Another commonly used example is early humans on the savannah. Rustling of the grass might indicate that a predator is near and be an indication to flee or get ready to fight, or it might be just the wind. Rapidly concluding that the rustle in the grass is a dangerous predator when it is actually just the wind doesn’t cost much, but believing that a dangerous predator is just the wind could easily mean that the early human ends up in the belly of a lion. The bottom line is that there likely was selective pressure in evolution against dismissing patterns (or for seeing them), whether they mean what we think they mean or not.

This pattern-seeking behavior is not well-suited to a task like evaluating drug safety. Many of us will conclude, after taking a drug or receiving a vaccine, that anything bad that happens soon afterward was likely caused by the drug or the vaccine, even when it is far more likely that it was random chance (i.e., coincidence), and nothing more. Such is the case with vaccines and autism. The ages during which children receive a lot of vaccines just happens to correspond to the ages where parents most frequently first start noticing the signs of autism spectrum disorders, as can be shown by an estimate of how many cases of autism we would expect to see diagnosed within one week or even 30 days of receiving vaccines. The numbers are much larger than most people would guess. Basically, our brains appear to be hard-wired to find patterns, and we have a hard time accepting that coincidence is very common. That’s why we need data and epidemiology to separate coincidence from actual correlation above what we would expect by random chance alone. It’s also why vaccine safety monitoring systems are so important.

Of course, I began expressing concerns about the likely problems President Trump would cause for science, especially drug approval and safety, beginning even before he took office and then during the first 100 days, such as his including two ultra-libertarians who thought that the FDA should only require evidence of safety and not efficacy before approving a drug, one of whom thought that voluntary reporting systems and the free market alone would be enough to assure the safety of drugs and devices, like a “Yelp” or “Uber” for pharmaceuticals, in his short list for candidates for FDA Commissioner. The third was a bona fide pharma shill, Scott Gottlieb. He was the “least bad” of the three and ultimately became FDA Commissioner. Surprisingly, he was generally not too bad, but unfortunately he only lasted a couple of years.

In any event, the Trump administration is focused on getting COVID-19 vaccines to market, but seems a lot less concerned about what comes next, such as how to distribute the vaccines rapidly and equitably, and how to monitor their longer term safety:

The next job will be to monitor the safety of vaccines once they’re in widespread use. But the administration last year quietly disbanded the office with the expertise for exactly this job, merging it into an office focused on infectious diseases. Its elimination has left that long-term safety effort for coronavirus vaccines fragmented among federal agencies, with no central leadership, experts say.

“We’re behind the eight ball,” said Daniel Salmon, who served as the director of vaccine safety in that office from 2007 to 2012, overseeing coordination during the H1N1 flu pandemic in 2009. “We don’t even know who’s in charge.”

An H.H.S. spokesperson said that the vaccine office was not shuttered. “The office was not ‘closed,’ but was merged with the Office of Infectious Disease and HIV/AIDS Policy and was strengthened,” the spokesperson said in a statement. “All the functions continue in this new organizational structure.”

In a brief statement, a different spokesperson said that Operation Warp Speed was working closely with the Centers for Disease Control and Prevention “to synchronize the IT systems” involved in monitoring vaccine safety data.

I’ll get into those IT systems in a minute, as I’ve written about them on multiple occasions. First, however, I note that the Office of Infectious Disease and HIV/AIDS Policy (OIDP) is an office in the Department of Health and Human Services (HHS) itself, not the CDC or FDA. A quick search of the almighty Wayback Machine at Archive.org reveals that the crawl of the web page listing the National Vaccine Program Office (NVPO) as part of OIDP is dated June 11, 2019, implying that the reorganization occurred sometime fairly soon before that. Searching for the NVPO further back on the Wayback Machine revealed web pages (i.e., this version from 2004) with much more robust links. Apparently the NVPO does have a Twitter feed, and it’s retweeting denials by Admiral Brett Giroir, Assistant Secretary for Health:

But is this true? Given the record of this administration compared to Carl Zimmer’s record as a journalist, I’d be far more inclined to believe Zimmer than Adm. Giroir, to be honest. But first, let’s look at the vaccine safety monitoring systems in place as they were pre-pandemic.

Beyond VAERS…

Antivaxxers love to portray the Vaccine Adverse Event Reporting System (VAERS) as the be-all and end-all of the databases monitoring vaccine safety. The reason is that VAERS is unique among US vaccine safety reporting systems in that it is a passive surveillance system. It relies on people to submit reports of adverse reactions to vaccines; it doesn’t actively look for them, as active surveillance systems do. Moreover, anyone can submit a report to VAERS, and they do, including parents of autistic children seeking compensation for their children’s autism as being due to “vaccine injury”. Indeed, I long ago discussed how lawyers game VAERS for their litigation, reporting lots of cases of autism as supposedly an “adverse reaction” to vaccines. It’s one reason why those of us who’ve been following the antivaccine movement like to refer to the bad “scientific studies” published by antivaccine physicians and scientists that use VAERS as their data source as “dumpster diving“, for example, this one claiming to find a link between the H1N1 vaccine and miscarriages or one of the earliest examples, Mark and David Geier’s epically bad study trying to link thimerosal-containing vaccines to autism. Meanwhile, in 2006 Jim Laidler infamously reported to VAERS that the flu vaccine had turned him into The Incredible Hulk, and VAERS accepted the report. True, someone did contact him to question it. If Laidler hadn’t been honest, he could have insisted that the report remain, and it would have.

Why would anyone set up a system like VAERS, which is co-managed by the Centers for Disease Control and Prevention (CDC) and the U.S. Food and Drug Administration (FDA)? First, you must understand that VAERS isn’t intended to give an accurate estimate of the frequency of various adverse events after vaccination. Rather, it was always intended to serve as an early warning system, a “canary in the coalmine”, so to speak. Consequently, even though antivaxxers like to harp on how passive surveillance systems generally capture only a small fraction of adverse reactions, one thing VAERS does do is capture severe reactions. Practically no one is going to report an adverse reaction like a sore arm or transient fever to VAERS, but you know damned well they’ll report more serious ones, such as a seizure. The problem is that the natural human tendency to seek patterns, coupled with the way antivaccine lawyers game VAERS by having their clients report all sorts of spurious “adverse events” to the database after vaccination, means that VAERS is a very noisy, distorted, and unreliable database. Still, it has its place if you understand its intended purpose. That purpose, contrary to what antivaxxers would have you believe, is not as the be-all and end-all of vaccine safety surveillance. It is primarily a hypothesis-generating, not hypothesis-testing, system when it comes to questions related to vaccine safety. To test the correlations found in VAERS requires different systems, systems such as the Vaccine Safety Datalink (VSD).

Antivaxxers sometimes mention VSD, but often labor under the delusion that it is not an automatic surveillance system. It is:

The Vaccine Safety Datalink (VSD) is a collaborative project between CDC’s Immunization Safety Office and eight health care organizations. The VSD started in 1990 and continues today in order to monitor safety of vaccines and conduct studies about rare and serious adverse events following immunization.

The VSD uses electronic health data from each participating site. This includes information on vaccines: the kind of vaccine given to each patient, date of vaccination, and other vaccinations given on the same day. The VSD also uses information on medical illnesses that have been diagnosed at doctors’ offices, urgent care visits, emergency department visits, and hospital stays. The VSD conducts vaccine safety studies based on questions or concerns raised from the medical literature and reports to the Vaccine Adverse Event Reporting System (VAERS). When there are new vaccines that have been recommended for use in the United States or if there are changes in how a vaccine is recommended, the VSD will monitor the safety of these vaccines.

So VSD is run by the CDC, meaning that it’s not part of this reorganization or shuttering or folding of NVPO into OIDP. Its strengths include being an active surveillance system that contains very granular data, given its tight affiliation with Kaiser Permanente in California, its Rapid Cycle Analysis (RCA), which allows it to identify adverse events following vaccination in near real time, and its data sharing and collaboration with other scientists outside of the CDC and the involved health care organizations. As an aside, it was an attempt to merge datasets from the VSD in a manner that would compromise the privacy of patients whose data were in the datasets that got antivaxxers Mark and David Geier into trouble so long ago.

Then, there are two active surveillance systems that cover vaccines. First, there is the Clinical Immunization Safety Assessment (CISA) project, in which the CDC partners with several large academic medical centers and Kaiser Permanente Northern California to study vaccine safety:

CISA addresses vaccine safety issues, conducts high quality clinical research, and assesses complex clinical adverse events following vaccination. CISA facilitates CDC’s collaboration with vaccine safety experts at leading academic medical centers and strengthens national capacity for vaccine safety monitoring. The CISA Project provides consultation to US clinicians who have vaccine safety questions about a specific patient residing in the US. In addition, CISA provides consultation to US healthcare providers and public health partners on vaccine safety issues, and reviews clinical adverse events following immunization (AEFI) involving the US-licensed vaccines.

Second, there is FDA’s Post-licensure Rapid Immunization Safety Monitoring System (PRISM):

The word “prism” might make you think of a triangular piece of glass that separates white light into a rainbow of colors.

But at FDA, it means a powerful, computer-based system that separates critical bits of information from vast streams of healthcare data in order to investigate adverse events and determine if there is a connection to a specific vaccine. And while the FDA prism–called Post-licensure Rapid Immunization Safety Monitoring System (PRISM)—might not have such a colorful name, it’s a bright light in the agency’s continual efforts to identify adverse effects in a timely manner.

PRISM is a cooperative effort between FDA’s Center for Biologics Evaluation and Research and its partners in the health care and medical insurance communities. It analyzes health insurance claims data from four national healthcare plans: Aetna, HealthCore (Wellpoint), Humana, and OptumInsight (United Healthcare).

And:

PRISM is one component of FDA’s Sentinel Initiative, which monitors the safety of a variety of FDA-regulated medical products by examining information in electronic healthcare databases.

Sentinel performs what is called “active” surveillance, as opposed to “passive” surveillance. Passive FDA surveillance systems depend on industry, consumers, patients, and healthcare professionals to recognize and report suspected adverse events to an FDA web site, such as the Vaccine Adverse Event Reporting System (VAERS). This means that FDA might not become aware of potential problems related to a licensed product for months.

Unlike passive surveillance, Sentinel’s active surveillance lets FDA initiate its own studies using existing electronic healthcare data in a timely manner. Sentinel also lets FDA evaluate safety issues in targeted groups, such as children, or to evaluate specific conditions (e.g., heart attacks) that are not usually reported as possible adverse events of medical products through passive reporting systems.

In summary, VSD uses real medical records, while PRISM and CISA are active surveillance systems. Contrary to what antivaxxers claim, there are at least four systems to monitor vaccine safety, two of which are active surveillance systems. Unfortunately, PRISM is no longer being used to its potential, as Zimmer reports.

What does the transfer of NVPO to OIDP mean?

Reading Zimmer’s article profoundly depressed me, because it just demonstrated to me yet again how little this administration cares for science. In particular, Zimmer mentioned the effective use of PRISM as a potential casualty of the transfer of NVPO to OIDP. First, Zimmer describes how PRISM came about:

In 1990, the C.D.C. set up a new way to track vaccines that didn’t depend on people coming forward. The agency worked with health care organizations to get updates on people’s medical conditions. That system now covers 12 million people. Researchers can use it to look for clusters of symptoms that arise in people who get the same vaccine.

When the H1N1 flu hit in 2009, Dr. Salmon recognized that these methods didn’t track enough people to quickly pick up rare symptoms. He reached out to researchers at Harvard to build a new system, which came to be known as PRISM. Ten states supplied vaccination records, and five health insurance companies shared anonymous information about 38 million members. PRISM then connected the two databases to track insurance claims in the wake of vaccination. “That really gave us a ton of data,” Dr. Salmon said.

The researchers could come up with a background rate of a host of medical conditions. If the H1N1 vaccine was linked to cases that matched the background rate, they could dismiss the symptoms as ordinary. Only if they rose above the background rate would they be considered unusual and warrant a closer look.

As Zimmer notes, the vast majority of reports turned out not to be correlated with vaccines, citing the example of some vaccinated people developing Bell’s palsy and how researchers were able to exclude vaccines as a cause within two weeks using PRISM. Now, for reasons that are unclear, PRISM appears to be no longer used to look for rare vaccine adverse events, the FDA using it only to monitor drug safety now:

The other systems can provide safety information much faster, but they’re small compared with the PRISM system, which now covers about 60 million people. The F.D.A. still uses PRISM for drug safety research, but not for vaccines. Dr. Salmon is baffled that the agency hasn’t tapped into it again. “Why would you not use that?” he asked. (An agency spokeswoman said it might use PRISM in the future should the need arise.)

Worse, though, is that there now appears to be a lack of coordination once provided by NVPO now that it has been folded into OIDP:

But Dr. Nicole Lurie, who was assistant secretary for preparedness and response at H.H.S. during the 2009 pandemic, said the loss of the vaccine safety office was especially costly once the coronavirus pandemic hit. “The coordinated leadership for stuff like this would likely come from the National Vaccine Program Office,” she said.

Dr. Lurie, now an adviser at the Coalition for Epidemic Preparedness Innovation, has been waiting along with other researchers, month after month, for coordinated leadership to emerge from the federal government on long-term vaccine safety. “There are a whole bunch of people who were really concerned about this,” she said.

An F.D.A. official who declined to be identified said that in the absence of the National Vaccine Program Office, F.D.A. and C.D.C. staff members were relying on relationships they had built across the agencies, meeting regularly to discuss their separate projects.

That leaderless effort concerns Dr. Lurie. “There’s no sort of active coordination to bring all the information together,” she said.

Given the track record of the Trump administration, I fear that scientists intensely interested in vaccine safety will have to wait a long time for coordinated leadership. If there’s one thing that is clear about this administration, even among those who support it, it’s that coordinated leadership really isn’t something it’s particularly good at.

Meanwhile, on Thursday an expert from the CDC and one from the FDA gave presentations about monitoring systems at the FDA’s vaccine advisory committee, describing one that will use smartphone apps to monitor health care workers and other essential workers after vaccination, with another looking at a database of insurance claims and electronic health records, with a third looking at data from the Centers for Medicare & Medicaid data to track people over 65 for adverse events after vaccination. None of these are as comprehensive as PRISM.

From my perspective, it’s hard for me to tell how much of this latest development in which the role of a key office for vaccine safety monitoring has apparently been much diminished in a “reorganization” (or whatever you want to call it) is due to incompetence or the hyperpoliticization of agencies that were intended to be apolitical. Could administration spokespeople claiming that NVPO is just as relevant and active as ever be correct? Who knows? I suppose it’s possible, but the statements of so many public health experts expressing alarm definitely give me pause.

Here’s what worries me, regardless of who wins the election. By the time COVID-19 vaccines are rolled out and distributed next year (at the earliest), the HHS, CDC, and FDA will have been “reorganized” and stripped of so much expertise through a brain drain resulting from the demoralization caused by so much incompetence and political interference that it will be very difficult for the CDC and FDA to monitor the safety of the new vaccines effectively. Even if Joe Biden wins, he can’t do anything before January 20, 2021, and it will take many months, if not several years to reverse the damage. As hard as it is to believe now, one day the pandemic will end. After that (no, even now) we have to worry about all the other routine vaccines, in particular how much confidence in vaccines will have eroded due to a distrust of the COVID-19 vaccine, the general incompetence and hostility to regulation of the current administration, and the unholy union of COVID-19 deniers, antivaxxers, and QAnon conspiracy theorists.